Note, of you approve the workflow, a comment like this one will be automatically created:

Hi @falkTX, I concluded this pull request.

Let me know if there are any missing here :)

I don't know why, but the step that add the code coverage didn't worked.

I will update this branch disabling this step, so the source will be tested, but if someone is interested in checking the coverage values it will be necessary to access the GitHub actions log. Anyway, the tests are the most important part 🙂

The step with error was disabled :)

ok getting back to this, there are a few things that I dont quite agree with.
main one being naming, calling the web request handler "controller" is very confusing as that is the name we give to the dwarf HMI side (aka "device controller").
having that being the namespace for both web requests and snapshot handling doesn't seem right to me either. ideally the web requests are decoupled from the state management requests.

I think the target split should be:

1. everything related to serving web content
2. state management
3. system tools and integration (not handled on this PR, just thinking out loud)

there is already a "session.py" middlewrap, which makes things even more confusing...

following what tornado does I think all the web related stuff should be under "web" name, this is what mod-ui code subclasses from anyhow.

while doing this we need to decide if we need to care about old duo/duox/dwarf compatibility, that still rely on python3.4
at this point it is not feasible to update the OS for these systems, as it is already known to work well. and from my experience updating buildroot and things in general, there is a lot of breakage happening...

so maybe this is the breaking point for existing MOD units, and past this point they dont get mod-ui updates? would be sad if so, specially on the Dwarf that is still being produced. if we care to keep compat with those, we need to find a way to test the code changes with python3.4 compat

with that said, do you mind if I start splitting some code into smaller pieces, following your initial ideas?

need to ask as that will quickly break this PR, needing to be redone. but it would be a way to get things moving...

with that said, do you mind if I start splitting some code into smaller pieces, following your initial ideas?

Ok, there is not a problem. Actually, I prefer. So I can see what direction you will path and I can follow it.

while doing this we need to decide if we need to care about old duo/duox/dwarf compatibility, that still rely on python3.4
at this point it is not feasible to update the OS for these systems, as it is already known to work well. and from my experience updating buildroot and things in general, there is a lot of breakage happening...

I prefer to maintain support. Related to this project, I would like to avoid any OS changes (dwarf included) to prevent to this project increases its scope. I think that after we refactor the backend, will be more clear what the Mod team can do related to old devices.

if we care to keep compat with those, we need to find a way to test the code changes with python3.4 compat

The main problem for creating tests there isn't the Python version. The main problem is the source code itself. So I believe that the code refactoring will make easier for creating tests.

The main problem is the source code itself.

I am sorry, maybe my comment was offensive, but it was not my intention. I tried to say that the source code is bit hard create tests and need some refactoring.

The main problem is the source code itself.

I am sorry, maybe my comment was offensive, but it was not my intention. I tried to say that the source code is bit hard create tests and need some refactoring.

no offense taken, source code doesnt have feelings you dont have to apologize to it.

PRs always go a bit slow, depends if there are other pending high-priority tasks happening at the moment. for better or worse it is indeed the case right now again. so dont expect any work on this at least until the NAMM show is over

sorry for lack of updates here. after some considerations we are delaying such a rework for a little bit until the next dwarf-supported release is done (so 1.14), but with good intentions.

we do not want to do a rework while needing to keep backwards compatibility with python3.4 as used for the dwarf (so I go back on what I said, yes) but at the same time there are nice features that are in the current code that just need a little polishing to make them usable. would be a bit wasteful to start a big rewrite now while those are not finished.
so here are the current plans:

1. finish the WIP stuff present on master branch (mostly about the drag&replace plugin feature)
2. release 1.14 as the "final" update for dwarf and other MOD units, keeping things in a "hotfix" as done for other releases
3. be now free to fully break the entire code and not have to care about backwards compatibility, we only need to care that it still works on the mod-desktop tool (so python3.8 compat because of mingw, but likely this could be updated too)
4. once everything works again and cleanup is done, slowly see if it is doable to bring the updated UI to MOD units, but if that takes 1 year or whatever doesn't matter so much anymore

so basically MOD will focus to get the next release out as a sorta final update using this current code-base, and then we won't bother with any new stuff for mod-ui side at all. any bugfixes can go into the "hotfix" branch.

tangent to this we want to start a separate project for a "state manager" that sits between mod-host and mod-ui.
instead of having mod-ui taking care of so much stuff (not just backend and frontend but also system operations, mixer etc etc), it should be split into separate components.
the intention of a "state manager" tool is to have that be the owner of the session, what is actively listening for connections (be it control chain, hmi, or other hw) and dispatches those to the needed points. acting as a "unique source of truth".
if this happens it will take at least a few months to materialize, but it shouldnt block the mod-ui rework. the rework will even help as it will allow to more easily swap mod-ui code to something managed elsewhere

for now the first thing to do is to finalize the drag&replace things, hopefully done in the next few weeks. please be on hold until that finishes, but discussion around the rework is still okay to take place of course.

Hello, @falkTX

Your idea remembers me so much my undergraduate project where I implemented something like these. There is a picture of the projects architecture:

• Plugins Manager offers a high-level API for plugins instantiation, plugins connection, etc: https://github.com/PedalPi/PluginsManager
  • It uses mod-host, but it's possible to use Carla.
  • I also did an experiment where I used Plugins Manager for controlling Zoom devices :)
• Application is an abstraction over Plugins Manager that add concepts like "current pedalboard", and persistence (for saving data): https://github.com/PedalPi/Application

Plugins Manager and Application are together the core that offers a single source of truth. And it is possible to extend it by the component's registration. Some examples of components:

• You can add a web-service REST if you want https://github.com/PedalPi/WebService (API preview)
• Or you can add an HMI controller for Raspberry https://pedalpi.github.io/blog/building-a-pedal-pi-component-pedalboard-selector.html
• Or maybe a serial communication using adb for android old devices https://github.com/PedalPi/WebServiceSerial

For instance, adding support for a MOD Device could be done at the same way, creating a component, where specific devices logic would be present only in the component source code.

You can read more checking the documentation and papers.

• https://pedalpi-application.readthedocs.io/en/latest/
• https://pedalpi-pluginsmanager.readthedocs.io/en/latest/
• https://www.researchgate.net/publication/337899259_Extending_Open-Source_Audio_Embedded_System_A_Pedal_Pi_Components_Study_Case
• https://www.researchgate.net/publication/331100984_Pedal_Pi_-_Multi-processador_de_plugins_de_audio_DIY

So, just complementing, because the mod functionalities are my main inspiration (mainly mod-host), the project has support to adding effects, removing effects, update parameters, connect audio ports and midi ports. But it is missing CV ports, for instance, because they were added later.

I don't know which language you would implement the single source of truth. But I humbly request you for see the upper code, maybe it could help your in some way.

for now the first thing to do is to finalize the drag&replace things, hopefully done in the next few weeks. please be on hold until that finishes, but discussion around the rework is still okay to take place of course.

ok, sure